Search for: All records

Abstract Trade-offs between producing costly movements for gathering information (‘explore’) and using previously acquired information to achieve a goal (‘exploit’) arise in a wide variety of problems, including foraging, reinforcement learning and sensorimotor control. Determining the optimal balance between exploration and exploitation is computationally intractable, necessitating heuristic solutions. Here we show that the electric fishEigenmannia virescensuses a salience-dependent mode-switching strategy to solve the explore–exploit conflict during a refuge-tracking task in which the same category of movement (fore-aft swimming) is used for both gathering information and achieving task goals. The fish produced distinctive non-Gaussian distributions of movement velocities characterized by sharp peaks for slower, task-oriented ‘exploit’ movements and broad shoulders for faster ‘explore’ movements. The measures of non-normality increased with increased sensory salience, corresponding to a decrease in the prevalence of fast explore movements. We found the same sensory salience-dependent mode-switching behaviour across ten phylogenetically diverse organisms, from amoebae to humans, performing tasks such as postural balance and target tracking. We propose a state-uncertainty-based mode-switching heuristic that reproduces the distinctive velocity distribution, rationalizes modulation by sensory salience and outperforms the classic persistent excitation approach while using less energy. This mode-switching heuristic provides insights into purposeful exploratory behaviours in organisms, as well as a framework for more efficient state estimation and control of robots. 

Abstract The transmission dynamics of HIV are closely tied to the duration and overlap of sexual partnerships. We develop an autonomous population model that can account for the possibilities of an infection from either a casual sexual partner or a long-term partner who was either infected at the start of the partnership or has been newly infected since the onset of the partnership. The impact of the long-term partnerships on the rate of infection is captured by calculating the expected values of the rate of infection from these extended contacts. The model includes three stages of infectiousness: acute, chronic, and virally suppressed. We calculate HIV incidence and the fraction of new infections attributed to casual contacts and long-term partnerships allowing for variability in condom usage, the effect of achieving and maintaining viral suppression, and early intervention by beginning HAART during the acute phase of infection. We present our results using data on MSM HIV transmission from the CDC in the U.S. While the acute stage is the most infectious, the majority of the new infections will be transmitted by long-term partners in the chronic stage when condom use is infrequent as is common in long-term relationships. Time series analysis of the solution, as well as parameter sensitivity analysis, are used to determine effective intervention strategies.